Cognitive heuristics in multitasking performance

A cognitive approach towards the understanding of human multitasking is presented in this contribution. Four dual-task studies provide evidence for the development and application of cognitive strategies which we refer to as “heuristics”: these fast and frugal rules of thumb help to adaptively use the information and structure of the environment to successfully perform two tasks concurrently. This work aims to show that heuristics not only arise unconsciously, but also change the selective visual attention by allowing different behavioral mechanisms (i.e., perception and action) to be performed in parallel. Interestingly, using the cognitive heuristics leads to an increase in performance. We used a realistic scenario in a driving simulator (study I and study II) and a systematically controlled and cognitive enriched tracking task in the laboratory (study III and study IV). Throughout all four studies, the results support our assumptions and encourage us to extend our research focus to other domains in the field of human machine interaction. Multitasking: A never-ending story In the 1920s, early approaches in the context of Gestalt Psychology investigated task interruption and dual task situations (Ovsiankina, 1928; Telford, 1931). Half a century ago, Cherry (1953) referred to “human’ s natural ability to multitask”. While in the last two decades, main interest was on the psychological refractory period (PRP-studies, see Pashler, 1994; Meyer & Kieras, 1997a, 1997b) or the central bottleneck – debate (Schumacher et al., 2001; Hazeltine et al., 2002; Anderson et al., 2005), the presented work is interested in the management of multiple goals in a real-life context. From a cognitive modeling perspective, multitasking can be considered as “the ability to integrate, interleave, and perform multiple tasks and/or component subtasks of a larger complex task” (Salvucci, 2005). Lee & Taatgen (2002) understand multitasking as “the ability to handle the demands of multiple tasks simultaneously”. Pew & Mavor (1998) simply refer to multitasking as “doing several things at once”. Our approach does not aspire a precise definition about what multitasking is. We rather intend to reveal the cognitive processing when doing several task at the same time. A taxonomic classification of dual task studies Before we introduce the applied dual task scenario, it is necessary to classify dual-task-situations. For real-life scenarios, Salvucci (2005) proposes four possible models that are worth introducing. Models of discrete successive tasks The first category specifies situations where we switch from one task to another. In empirical investigations, this is often applied in alternating trials of simple choice-reaction tasks. Main interest thus is paid in respect of temporal costs for switching (Rogers & Monsell, 1995). Models of discrete concurrent tasks PRP-studies (Pashler, 2000) belong to the second category. Offset by a short delay, a second task starts before the first one ends. This delay leads to a short overlapping of the two applied tasks. Models of elementary continuous tasks The third category allows one continuous task (most experimental scenarios use tracking as main task). Short discrete tasks occasionally appear and thus initiate task concurrency. Models of compound continuous tasks If two or more simultaneous tasks appear, we talk about models of compound continuous tasks: each of the both tasks is an “ongoing continuous process”. In our eyes, this last category implies a high degree on ecological validity and reflects human behavior in typical real-life-situations. Especially in the context of human machine interaction, we often have to deal with the concurrency of two (and sometimes even more) tasks. We therefore consider the presented four empirical investigations as models of compound continuous tasks. Figure 1: Multiple resource model (Wickens, 2002) The boundedness of human cognitive resource In contrast to assumptions about one single resource to be responsible for the management of human resources (Kahneman, 1992), the reference frame for our studies is the four-dimensional multiple resource model proposed by Wickens (2002, 2004): processing stages (perception, cognition, responding), perceptual modalities (auditory, visual), visual channels (focal, ambient), and processing codes (spatial, verbal) rely on different physiological mechanism (Fig. 1). Following this model, two tasks do not influence each other and hence can be executed in parallel if they require different resources. Figure 2: In-car-scenario (Driving simulator) Empirical framework, Part A: Study I & II In four experiments with scenarios of compound continuous tasks, we analyze the behaviour in dynamical task environments. Main task in all studies was driving: in a simple tracking task in a driving simulator, students of the TU Berlin (aged 20 to 40) were instructed to keep the lane (Fig. 2). Study one: Identification of cognitive strategies 24 participants joined the first study. Driving was instructed as priority task, secondary task was a modification of the D2 test of attention by Brickenkamp (2001). We refer to this implemented in-car-version as D2-Drive (Urbas et al., 2005): goal of this test is to verify whether a specific pattern contains the letter d and additionally two strokes (yesresponse) or whether not (no-response). Fig. 3 illustrates the three versions: subjects performed either the pattern in the middle (version A), or the complete line (version B and C). Please note that version C also contains a memory task: the number of the line to be performed on the next screen had to be memorized. The D2-Drive test measures the individual amount of visual attention, it can be interrupted and easily be learned. This makes it a perfect candidate to identify performance strategies under multitasking. Figure 3: D2-Drive test (Urbas et al., 2005) The sequence of the complete study was as follows: after a training period (for driving as well as for the D2-Drive test), both tasks were presented in single task and dual task condition. Task complexity in the secondary task (Fig. 3) was treated as within-subject variable. Dependent variables were driving performance (lane derivation) and D2-Driveperformance (number of correct patterns and performance time). Eye-tracking data was recorded in all possible conditions. After each test episode, workload was measured using the NASA TLX (Hart & Staveland, 1988). The study ended with a structured interview in which subjects were asked about general task processing, perceived task complexity and applied cognitive strategies. Core results of the first study After a short training, subjects performed the D2-Drive test error-free. Under dual task condition, driving performance did not decrease. It is worth mentioning that performance in D2-Drive did increase slightly under multitasking from trial to trial. In particular, this is true for version B: here, the task configuration promotes the merging of several patterns. This approach is confirmed by the structured interviews and justifies a deeper analysis of the strategic processing. Cognitive strategies under multitasking Starting with the D2-Drive test under single task condition, subjects performed pattern by pattern. This stepwise approach goes in line with people doing a task for the first time, as described by Taatgen (2005). Under dual task condition (and for some people, this is even true at the end of the single task condition), subjects intuitively understand that part of the secondary task (more precisely, the respondaction) does not require focal visual attention, but only manual action (Fig. 4). Manual and visual activities are decoupled: when entering the sequence of responses (from 2 up to 4 patterns), the visual channel is free. Consequently, during this time subjects are able to afford a control view at the street. This finding goes in line with the theoretical assumptions derived by the Wickens’ model of multiple resources (Wickens, 2002). Figure 4: Development of a cognitive strategy Please note that subjects do not necessarily have conscious access to the described cognitive strategies, as indicated by the structured interviews. For this reason, we refer to them as “cognitive heuristics”: heuristics are replicable methods often discovered in the field of complex problem solving or learning and provide a high efficiency in performance. In this article, the words “strategy” and “heuristic” are treated synonymously, even though the term “heuristic” captures a broader sense. To verify the use of cognitive heuristics such as the one example represented in Fig. 4, we did a second study in the driving simulator. Study two: The influence of experience Goal of the second study was to confirm the results of the first study. Despite the rather weak increase in performance, we expected a more pronounced effect if subjects are more experienced in a concrete multitasking situation. This is derived from studies reported by Taatgen (2005) as well as Salvucci & Taatgen (submitted). Consequently the dual task conditions were temporarily doubled, each subject attended two driving laps. Figure 5: Performance in the D2-Drive test in study II Performance increase as a result of experience As expected, performance increased under multitasking for both version A (slightly significant, p < .05) and version B (high significant, p < .01). Interestingly, even in version C, there is no decrease in performance (Fig. 5). Although observational data (eye tracking) as well as interviews support the proposed cognitive heuristics, we decided to formalize our hypotheses and thus make it revisable. To do so, we used the method of cognitive modeling. The benefit of Cognitive Modeling: Psychological research in general starts with a theory, for instance on human multitasking (see Salvucci, 2006). The derived hypotheses (assumptions) subsequently are tested (empirical investigation): results of the experiments support or reject the assumptions and thus the theory. Going one step further, the theory-based assumption can precisely be formalized in a cognitive model (Fig. 6). We used the ACTR framework (Anderson et al., 2004). The model-based predictions are compared with the empirical results, and the degree of matching constitutes the value of the cognitive model. Figure 6: Cognitive modeling, in a nutshell In the first two studies, the same versions of the D2-Drive test were applied. While doing this test, two different levels of performance can be highlighted: on a micro-level (i.e., reading a single pattern), participants’ visual scan paths give insight into how one generous test element is treated. This process is similar to reading. For almost all participants, their “reading style” developed over time: whilst beginning as expected (“read” complete pattern, e.g. middle-upperlower area), most people realize already after a short time that for p-patterns (i.e., patterns containing the letter p), it is sufficient to stop after checking the middle part. Such a processing is a reasonable short-cut to save important time. We call it “micro-strategy” and distinguish it from a stepwise processing on a macro-level: the sequential processing (i.e., visual orientation – read first pattern – react – read second pattern – look at street – etc.) is called “macro-strategy”, the successfully repeated use of a concrete macro-strategy is treated as a cognitive heuristic. Figure 7:Empirical results vs. model-based predictions Both microas well as macro-strategies were implemented in ACT-R (Anderson et al., 2004; Taatgen, 1999): a comparison between corresponding models (simulation) and empirical results (experiments) in all cases highly support the claim that subjects use cognitive heuristics within the structure of the given task environment. Empirical framework, Part B: Study III & IV The first two studies concentrated on the basic mechanisms of human multitasking in a realistic task scenario. Our aim was to verify the cognitive processing while performing two concurrent tasks. Although we achieved a realistic multitasking scenario, the primary task (driving in terms of simple tracking) turned out to be “too” simple: we therefore decided to apply a more complex driving task which at the same time should be systematically controlled. Study III and study IV incorporate this issue: we used the lane change task (LCT, Mattes, 2003) in a standardized PC-based version (Fig. 5): in this test, participants first see a sign (perception) which tells them to which position they must change the lane (e.g., to the right lane). In a second step, participants decide what to do and change the lane accordingly (reaction) which constitutes step three (i.e., the lane change maneuver). Finally, they have to keep the “new” lane until a next sign appears. This primary task was used in both study III and study IV. TASK: Change the lane as soon as you see the sign 2. React accordingly 4. Keep the new lane 3. Lane change 1. Sign perception Figure 5: Lane change task (Mattes, 2003) Study three: The impact of task complexity Task complexity in the primary task was obtained by applying the LCT (i.e., an additional cognitive component). The first two studies suggest that version B of the D2-Drive test is convenient for investigating multitasking heuristics due to its structure. We therefore created four different version (see Table 1) to testify whether a task’ s degrees of freedom generates the postulated strategic behavior. Table 1: Complexity of the secondary task Version Task description D2-Drive-1 Performing complete line of five patterns (see version D2-Drive-B in study I and study II). D2-Drive-2 Same as D2-Drive-1 (performing from position 1 up to 5), but after each response, the complete row changes. D2-Drive-3 Same as D2-Drive-1, but enhanced by additional marker to show current position. D2-Drive-4 Combination of D2-Drive-2 and D2Drive-3: patterns dynamically change after each response, but current position is highlighted by visual marker. Table 1 specifies the four different version in detail: D2Drive-1 and D2-Drive-3 contain a fixed row of five patterns, in D2-Drive-1 and D2-Drive-3, the row changes after each manual action and five new patterns are presented. Version 1 and 3 allow to look ahead and merge together several patterns (as described in Fig. 4), whereas version 2 and 4 do not contain this option. Version 3 and 4 share as common feature a “visual eye”, i.e. a marker that highlights the current position. This is a useful assistance when switching back to D2-Drive. Version 1 and 2 do not provide such a helping mechanism for visual orientation. We expected a higher amount of strategy use for version 1 and D2-Drive-2, but not for 3 and 4 due to forced focusing. We further assumed D2-Drive-1 and D2-Drive-3 to support the use of cognitive heuristics as a consequence of possible anticipating. Detailed results of primary as well as secondary tasks can be found elsewhere (Kiefer & Urbas, 2006). Here, it is solely mentioned that (1) participants’ reports on cognitive heuristics mirrored those of the first two studies. In combination with eye movement data, the additional (visual) marker splits all participants into two groups, namely those who are supported by “visual help” (22 out of 40) and those who experience the task as more complicated (14 out of 40). Four people felt neutral (this was also confirmed by their performance data). Study four: The role of time pressure The first two studies show that experience and training highly influences the development and application of cognitive heuristics. Even under cognitive demanding primary tasks (study III), subjects are able to work successfully with the concept of cognitive heuristics. But one main factor influencing the application of a heuristic in a concrete situation seems to be the task configuration. Study IV is an extension of study III insofar as it enriches the scenario by the factor “time pressure”. As mentioned before, experience plays a key role in the development and application of cognitive heuristics under multitasking. Experience is often promoted by time pressure: under high time pressure, experienced behavior exposes earlier due to temporal constraints. Study IV consequently investigates the impact of time pressure in dual task situations. We applied two versions of the D2-Drive test: one version in which participants are able to apply cognitive heuristics by merging elements together, a second version does not allow this processing (see D2-Drive-2 in study III). First results confirm the importance of our consideration: time pressure does not intensively influence performance in D2-Drive but plays a key role for cognitive heuristics. People almost immediately seem to understand how the task configuration (LCT and D2-Drive) can be optimized and act accordingly. Please not further that even though the factor time pressure was directed toward both tasks, its main impact was on the secondary task. We assume that the primary task is too rigid and performance thus cannot be optimized in terms of applying strategic ideas compared to the D2-Drive test. Conclusion and outlook Four studies describe the current state of our effort to understand human behaviour in complex multitasking situations. We conclude the main results and explicate forthcoming investigations.